Reinforcement Learning in Control

During its melt cycle, an arc furnace causes disturbances of the electrical supply. Existing measurement techniques for this application lead to corrective rather than predictive compensation. The use of neural networks to control the compensation is being considered, in particular reinforcement learning strategies which require no pre-training and which can adapt to a dynamically changing environment. Several reinforcement learning techniques have been considered by examining their effectiveness in learning to balance a pole on a moving cart without prior training. The network is required to produce an appropriate control action in response to the current world state in order to maintain the pole and cart position within acceptable limits. One reason for investigating this is the belief that many of the characteristics of the pole-balancer are analogous to the problem of compensating for reactive power disturbances in the arc furnace. This paper presents a comparative review of these reinforcement learning strategies.